Evaluating defect prediction approaches: a benchmark and an extensive comparison

Reliably predicting software defects is one of the holy grails of software engineering. Researchers have devised and implemented a plethora of defect/bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches. We present a benchmark for defect prediction, in the form of a publicly available dataset consisting of several software systems, and provide an extensive comparison of well-known bug prediction approaches, together with novel approaches we devised. We evaluate the performance of the approaches using different performance indicators: classification of entities as defect-prone or not, ranking of the entities, with and without taking into account the effort to review an entity. We performed three sets of experiments aimed at (1) comparing the approaches across different systems, (2) testing whether the differences in performance are statistically significant, and (3) investigating the stability of approaches across different learners. Our results indicate that, while some approaches perform better than others in a statistically significant manner, external validity in defect prediction is still an open problem, as generalizing results to different contexts/learners proved to be a partially unsuccessful endeavo

On porting software visualization tools to the web

Software systems are hard to understand due to the complexity and the sheer size of the data to be analyzed. Software visualization tools are a great help as they can sum up large quantities of data in dense, meaningful pictures. Traditionally, such tools come in the form of desktop applications. Modern web frameworks are about to change this status quo, as building software visualization tools as web applications can help in making them available to a larger audience in a collaborative setting. Such a migration comes with a number of promises, perils, and technical implications that must be considered before starting any migration process. In this paper, we share our experiences in porting two such tools to the web and provide guidelines about the porting. In particular, we discuss promises and perils that go hand in hand with such an endeavor and present a number of technological alternatives that are available to implement web-based visualization

On the evolution of source code and software defects

Software systems are subject to continuous changes to adapt to new and changing requirements.This phenomenon, known as software evolution, leads in the long term to software aging: The size and the complexity of systems increase, while their quality decreases. In this context, it is no wonder that software maintenance claims the most part of a software system's cost.The analysis of software evolution helps practitioners deal with the negative effects of software aging. With the advent of the Internet and the consequent widespread adoption of distributed development tools, such as software configuration management and issue tracking systems, a vast amount of valuable information concerning software evolution has become available. In the last two decades, researchers have focused on mining and analyzing this data, residing in various software repositories, to understand software evolution and support maintenance activities. However, most approaches target a specific maintenance task, and consider only one of the several facets of software evolution. Such approaches, and the infrastructures that implement them, cannot be extended to address different maintenance problems. In this dissertation, we propose an integrated view of software evolution that combines different evolutionary aspects. Our thesis is that an integrated and flexible approach supports an extensible set of software maintenance activities. To this aim, we present a meta-model that integrates two aspects of software evolution: source code and software defects. We implemented our approach in a framework that, by retrieving information from source code and defect repositories, serves as a basis to create analysis techniques and tools. To show the flexibility of our approach, we extended our meta-model and framework with e-mail information extracted from development mailing lists. To validate our thesis, we devised and evaluated, on top of our approach, a number of novel analysis techniques that achieve two goals: 1. Inferring the causes of problems in a software system. We propose three retrospective analysis techniques, based on interactive visualizations, to analyze the evolution of source code, software defects, and their co-evolution. These techniques support various maintenance tasks, such as system restructuring, re- documentation, and identification of critical software components. 2. Predicting the future of a software system. We present four analysis techniques aimed at anticipating the locations of future defects, and investigating the impact of certain source code properties on the presence of defects. They support two maintenance tasks: defect prediction and software quality analysis. By creating our framework and the mentioned techniques on top of it, we provide evidence that an integrated view of software evolution, combining source code and software defects information, supports an extensible set of software maintenance tasks

Analyzing software repositories to understand software evolution

Software repositories such as versioning systems, defect tracking systems, and archived communication between project personnel are used to help manage the progress of software projects. Software practitioners and researchers increasingly recognize the potential benefit of mining this information to support the maintenance of software systems, improve software design or reuse, and empirically validate novel ideas and techniques. Research is now proceeding to uncover ways in which mining these repositories can help to understand software development, to support predictions about software development, and to plan various evolutionary aspects of software projects. This chapter presents several analysis and visualization techniques to understand software evolution by exploiting the rich sources of artifacts that are available. Based on the data models that need to be developed to cover sources such as modification and bug reports we describe how to use a Release History Database for evolution analysis. For that we present approaches to analyze developer effort for particular software entities. Further we present change coupling analyses that can reveal hidden change dependencies among software entities. Finally, we show how to investigate architectural shortcomings over many releases and to identify trends in the evolution. Kiviat graphs can be effectively used to visualize such analysis results